A generalized characterization of algorithmic probability

An a priori semimeasure (also known as "algorithmic probability" or "the Solomonoff prior" in the context of inductive inference) is defined as the transformation, by a given universal monotone Turing machine, of the uniform measure on the infinite strings. It is shown in this paper that the class of a priori semimeasures can equivalently be defined as the class of transformations, by all compatible universal monotone Turing machines, of any continuous computable measure in place of the uniform measure. Some consideration is given to possible implications for the prevalent association of algorithmic probability with certain foundational statistical principles

Putnam's Diagonal Argument and the Impossibility of a Universal Learning Machine

The diagonalization argument of Putnam (1963) denies the possibility of a universal learning machine. Yet the proposal of Solomonoff (1964) and Levin (1970) promises precisely such a thing. In this paper I discuss how their proposed measure function manages to evade Putnam's diagonalization in one respect, only to fatally fall prey to it in another

Universal Prediction

In this thesis I investigate the theoretical possibility of a universal method of prediction. A prediction method is universal if it is always able to learn from data: if it is always able to extrapolate given data about past observations to maximally successful predictions about future observations. The context of this investigation is the broader philosophical question into the possibility of a formal specification of inductive or scientific reasoning, a question that also relates to modern-day speculation about a fully automatized data-driven science. I investigate, in particular, a proposed definition of a universal prediction method that goes back to Solomonoff (1964) and Levin (1970). This definition marks the birth of the theory of Kolmogorov complexity, and has a direct line to the information-theoretic approach in modern machine learning. Solomonoff's work was inspired by Carnap's program of inductive logic, and the more precise definition due to Levin can be seen as an explicit attempt to escape the diagonal argument that Putnam (1963) famously launched against the feasibility of Carnap's program. The Solomonoff-Levin definition essentially aims at a mixture of all possible prediction algorithms. An alternative interpretation is that the definition formalizes the idea that learning from data is equivalent to compressing data. In this guise, the definition is often presented as an implementation and even as a justification of Occam's razor, the principle that we should look for simple explanations. The conclusions of my investigation are negative. I show that the Solomonoff-Levin definition fails to unite two necessary conditions to count as a universal prediction method, as turns out be entailed by Putnam's original argument after all; and I argue that this indeed shows that no definition can. Moreover, I show that the suggested justification of Occam's razor does not work, and I argue that the relevant notion of simplicity as compressibility is already problematic itself

The Meta-Inductive Justification of Induction: The Pool of Strategies

This paper poses a challenge to Schurz's proposed meta-inductive justification of induction. It is argued that Schurz's argument requires a notion of optimality that can deal with an expanding pool of prediction strategies

Universal Prediction

In this dissertation I investigate the theoretical possibility of a universal method of prediction. A prediction method is universal if it is always able to learn what there is to learn from data: if it is always able to extrapolate given data about past observations to maximally successful predictions about future observations. The context of this investigation is the broader philosophical question into the possibility of a formal specification of inductive or scientific reasoning, a question that also touches on modern-day speculation about a fully automatized data-driven science. I investigate, in particular, a specific mathematical definition of a universal prediction method, that goes back to the early days of artificial intelligence and that has a direct line to modern developments in machine learning. This definition essentially aims to combine all possible prediction algorithms. An alternative interpretation is that this definition formalizes the idea that learning from data is equivalent to compressing data. In this guise, the definition is often presented as an implementation and even as a justification of Occam's razor, the principle that we should look for simple explanations. The conclusions of my investigation are negative. I show that the proposed definition cannot be interpreted as a universal prediction method, as turns out to be exposed by a mathematical argument that it was actually intended to overcome. Moreover, I show that the suggested justification of Occam's razor does not work, and I argue that the relevant notion of simplicity as compressibility is problematic itself

Universal Prediction:A Philosophical Investigation

In dit proefschrift onderzoek ik de theoretische mogelijkheid van een universele methode van voorspelling. Een voorspelmethode is universeel als deze altijd in staat is te leren van invoergegevens: als deze altijd in staat is gegevens over eerdere observaties door te trekken naar maximaal succesvolle voorspellingen over toekomstige observaties. De context van dit onderzoek is de bredere filosofische vraag naar de mogelijkheid van een formele specificatie van inductief of wetenschappelijk redeneren, een vraag die raakt aan hedendaagse speculatie over een volledig geautomatiseerde datagedreven wetenschap. Meer bepaald onderzoek ik een specifieke wiskundige definitie van een universele voorspelmethode, die opgesteld werd in de begindagen van de kunstmatige intelligentie en die een directe lijn heeft naar moderne ontwikkelingen in machinaal leren. Deze definitie is in wezen een poging alle mogelijke voorspelalgoritmes samen te voegen. Een alternatieve interpretatie is dat deze definitie een formalisering geeft van het idee dat leren uit gegevens equivalent is aan het comprimeren ervan. In deze hoedanigheid wordt de definitie ook wel voorgesteld als een implementatie en zelfs een rechtvaardiging van Occams scheermes, het principe dat we moeten streven naar eenvoudige verklaringen. De bevindingen van mijn onderzoek zijn negatief. Ik toon aan dat de onderzochte definitie niet geïnterpreteerd kan worden als een universele voorspelmethode, zoals blijkt te volgen uit een wiskundig argument dat het juist bedoeld was te ontwijken. Bovendien laat ik zien dat de gesuggereerde rechtvaardiging van Occams scheermes niet opgaat, en beargumenteer ik dat de relevante notie van eenvoud als comprimeerbaarheid zelf problematisch is

Solomonoff Prediction and Occam's Razor

Algorithmic information theory gives an idealized notion of compressibility, that is often presented as an objective measure of simplicity. It is suggested at times that Solomonoff prediction, or algorithmic information theory in a predictive setting, can deliver an argument to justify Occam's razor. This paper explicates the relevant argument, and, by converting it into a Bayesian framework, reveals why it has no such justificatory force. The supposed simplicity concept is better perceived as a specific inductive assumption, the assumption of effectiveness. It is this assumption that is the characterizing element of Solomonoff prediction, and wherein its philosophical interest lies

The Meta-Inductive Justification of Induction

I evaluate Schurz's proposed meta-inductive justification of induction, a refinement of Reichenbach's pragmatic justification that rests on results from the machine learning branch of prediction with expert advice. My conclusion is that the argument, suitably explicated, comes remarkably close to its grand aim: an actual justification of induction. This finding, however, is subject to two main qualifications, and still disregards one important challenge. The first qualification concerns the empirical success of induction. Even though, I argue, Schurz's argument does not need to spell out what inductive method actually consists in, it does need to postulate that there is something like the inductive or scientific prediction strategy that has so far been *significantly* more successful than alternative approaches. The second qualification concerns the difference between having a justification for inductive method and for sticking with induction *for now*. Schurz's argument can only provide the latter. Finally, the remaining challenge concerns the pool of alternative strategies, and the relevant notion of a meta-inductivist's optimality that features in the analytical step of Schurz's argument. Building on the work done here, I will argue in a follow-up paper that the argument needs a stronger *dynamic* notion of a meta-inductivist's optimality